A Sarsa( λ

Solving reinforcement learning problems in continuous space with function approximation is currently a research hotspot of machine learning. When dealing with the continuous space problems, the classic Q-iteration algorithms based on lookup table or function approximation converge slowly and are difficult to derive a continuous policy. To overcome the above weaknesses, we propose an algorithm named DFR-Sarsa(λ) based on double-layer fuzzy reasoning and prove its convergence. In this algorithm, the first reasoning layer uses fuzzy sets of state to compute continuous actions; the second reasoning layer uses fuzzy sets of action to compute the components of Q-value. Then, these two fuzzy layers are combined to compute the Q-value function of continuous action space. Besides, this algorithm utilizes the membership degrees of activation rules in the two fuzzy reasoning layers to update the eligibility traces. Applying DFR-Sarsa(λ) to the Mountain Car and Cart-pole Balancing problems, experimental results show that the algorithm not only can be used to get a continuous action policy, but also has a better convergence performance

Efficient Actor-Critic Algorithm with Hierarchical Model Learning and Planning

To improve the convergence rate and the sample efficiency, two efficient learning methods AC-HMLP and RAC-HMLP (AC-HMLP with l2-regularization) are proposed by combining actor-critic algorithm with hierarchical model learning and planning. The hierarchical models consisting of the local and the global models, which are learned at the same time during learning of the value function and the policy, are approximated by local linear regression (LLR) and linear function approximation (LFA), respectively. Both the local model and the global model are applied to generate samples for planning; the former is used only if the state-prediction error does not surpass the threshold at each time step, while the latter is utilized at the end of each episode. The purpose of taking both models is to improve the sample efficiency and accelerate the convergence rate of the whole algorithm through fully utilizing the local and global information. Experimentally, AC-HMLP and RAC-HMLP are compared with three representative algorithms on two Reinforcement Learning (RL) benchmark problems. The results demonstrate that they perform best in terms of convergence rate and sample efficiency

Identification of membrane protein types via deep residual hypergraph neural network

A membrane protein's functions are significantly associated with its type, so it is crucial to identify the types of membrane proteins. Conventional computational methods for identifying the species of membrane proteins tend to ignore two issues: High-order correlation among membrane proteins and the scenarios of multi-modal representations of membrane proteins, which leads to information loss. To tackle those two issues, we proposed a deep residual hypergraph neural network (DRHGNN), which enhances the hypergraph neural network (HGNN) with initial residual and identity mapping in this paper. We carried out extensive experiments on four benchmark datasets of membrane proteins. In the meantime, we compared the DRHGNN with recently developed advanced methods. Experimental results showed the better performance of DRHGNN on the membrane protein classification task on four datasets. Experiments also showed that DRHGNN can handle the over-smoothing issue with the increase of the number of model layers compared with HGNN. The code is available at https://github.com/yunfighting/Identification-of-Membrane-Protein-Types-via-deep-residual-hypergraph-neural-network

The genome of broomcorn millet

Broomcorn millet (Panicum miliaceum L.) is the most water-efficient cereal and one of the earliest domesticated plants. Here we report its high-quality, chromosome-scale genome assembly using a combination of short-read sequencing, single-molecule real-time sequencing, Hi-C, and a high-density genetic map. Phylogenetic analyses reveal two sets of homologous chromosomes that may have merged ~5.6 million years ago, both of which exhibit strong synteny with other grass species. Broomcorn millet contains 55,930 proteincoding genes and 339 microRNA genes. We find Paniceae-specific expansion in several subfamilies of the BTB (broad complex/tramtrack/bric-a-brac) subunit of ubiquitin E3 ligases, suggesting enhanced regulation of protein dynamics may have contributed to the evolution of broomcorn millet. In addition, we identify the coexistence of all three C4 subtypes of carbon fixation candidate genes. The genome sequence is a valuable resource for breeders and will provide the foundation for studying the exceptional stress tolerance as well as C4 biology

Unveiling the additive-assisted oriented growth of perovskite crystallite for high performance light-emitting diodes.

Solution-processed metal halide perovskites have been recognized as one of the most promising semiconductors, with applications in light-emitting diodes (LEDs), solar cells and lasers. Various additives have been widely used in perovskite precursor solutions, aiming to improve the formed perovskite film quality through passivating defects and controlling the crystallinity. The additive's role of defect passivation has been intensively investigated, while a deep understanding of how additives influence the crystallization process of perovskites is lacking. Here, we reveal a general additive-assisted crystal formation pathway for FAPbI3 perovskite with vertical orientation, by tracking the chemical interaction in the precursor solution and crystallographic evolution during the film formation process. The resulting understanding motivates us to use a new additive with multi-functional groups, 2-(2-(2-Aminoethoxy)ethoxy)acetic acid, which can facilitate the orientated growth of perovskite and passivate defects, leading to perovskite layer with high crystallinity and low defect density and thereby record-high performance NIR perovskite LEDs (~800 nm emission peak, a peak external quantum efficiency of 22.2% with enhanced stability)

Building Energy Consumption Prediction Using a Deep-Forest-Based DQN Method

When deep reinforcement learning (DRL) methods are applied in energy consumption prediction, performance is usually improved at the cost of the increasing computation time. Specifically, the deep deterministic policy gradient (DDPG) method can achieve higher prediction accuracy than deep Q-network (DQN), but it requires more computing resources and computation time. In this paper, we proposed a deep-forest-based DQN (DF–DQN) method, which can obtain higher prediction accuracy than DDPG and take less computation time than DQN. Firstly, the original action space is replaced with the shrunken action space to efficiently find the optimal action. Secondly, deep forest (DF) is introduced to map the shrunken action space to a single sub-action space. This process can determine the specific meaning of each action in the shrunken action space to ensure the convergence of DF–DQN. Thirdly, state class probabilities obtained by DF are employed to construct new states by considering the probabilistic process of shrinking the original action space. The experimental results show that the DF–DQN method with 15 state classes outperforms other methods and takes less computation time than DRL methods. MAE, MAPE, and RMSE are decreased by 5.5%, 7.3%, and 8.9% respectively, and R2 is increased by 0.3% compared to the DDPG method

Habitat Niche-Fitness and Radix Yield Prediction Models for Angelica sinensis Cultivated in the Alpine Area of the Southeastern Region of Gansu Province, China

Dried root of Angelica sinensis has been used for thousands of years in traditional Chinese medicinal prescriptions. Researches on better knowledge of appropriate habitats for cultivation of this species are required to encourage the potential ecological sustainable industry. From 2001 to 2004, transplanting trials on the regulation of fertilizers and planting density were conducted for collection of habitat factor data at four sites of four counties in the southeastern region of Gansu Province, China. Introducing the niche theory into the research, habitat niche-fitness (HNF) is defined as the degree of similarity of an actual habitat state to the optimum habitat. A new model of HNF is constructed to evaluate the adaptive extent of A. sinensis. The results showed that the model of HNF notably outperforms the proportional similarity index and the geometric parallelism formula both in mathematical justification and biological principle testing. With HNF as a surrogate for composite environmental factors, a radix yield model was constructed. Evaluation of the present model by the sampled subplots specified for data validation proved that the model could be well used for predicted of radix yield across a wide-spread area. The radix yield prediction model and its uses are recommended within the limitations of the data used in the study area. Beyond this range, validation of the radix yield prediction model will be necessary

Network Security Situation Prediction Model Based on VMD Decomposition and DWOA Optimized BiGRU-ATTN Neural Network

The widespread adoption of Internet-of-Things (IoT) devices has resulted in a comprehensive transformation of human life. However, the network security challenges posed by the IoT devices have become increasingly severe, necessitating the implementation of effective security mechanisms. Network security situational awareness enables an effective network state prediction for better formulation of network security defense strategies. Existing network security situational prediction methods are typically constrained by situational sequence data, especially those sequences with a high degree of non-stationarity, leading to unstable predictions and low performance. Moreover, in real-world application scenarios, the network security situational sequences are often highly non-stationary. To address these challenges, we introduce a novel hybrid prediction model named Variational Mode Decomposition (VMD) - Dynamic Whale Optimization Algorithm (DWOA) - Bidirectional Gated Recurrent Unit (BiGRU) - Attention Mechanism (ATTN). The proposed model integrates VMD, BiGRU, ATTN, and DWOA. Initially, network security situational awareness sequences are processed using VMD to decompose them into a series of subsequences, thus reducing the non-stationarity of the original sequences. Subsequently, an enhanced DWOA optimization algorithm is introduced for tuning the hyperparameters of the BiGRU-ATTN network. Ultimately, BiGRU-ATTN is employed to predict each of these subsequences, which are then aggregated to yield the final network security situational prediction value. When compared with several existing methods on public network security datasets, the proposed VMD-DWOA-BiGRU-ATTN method demonstrated an improvement in the R^2 values ranging from 6.34% to 52.61%. These results substantiate that the model significantly enhances predictive performance